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AIR-TO-AIR  COMBAT  SKILLS:  CONTRIBUTION  OF 
PLATFORM  MOTION  TO  INITIAL  TRAINING 


I.  INTRODUCTION 

General  Background 

Until  recently,  the  need  for  platform  motion  on 
aircraft  simulation  devices  was  largely  accepted  on 
an  intuitive  basts.  It  was  felt  that  some  degree  of 
motion  enhanced  the  “realism"  of  the  simulation 
and  thus  provided  more  effective  training.  For  the 
most  part,  both  the  Air  Force  and  the  commercial 
airline  industry  have  maintained  this  posture  for 
some  time.  Recently,  however,  this  traditional 
posture  has  been  questioned  from  both  training 
and  cost  viewpoints.  The  essential  training 
question  is  whether  the  transfer  of  training  from 
simulator  to  aircraft  is  enhanced  by  simulator 
platform  motion.  Procurement,  facility,  and 
operation  and  maintenance  costs  vary  substantially 
as  a function  of  the  inclusion  or  exclusion  of 
platform  motion.  Because  of  the  number  of 
planned  Air  Force  simulator  procurements,  wise 
investment  strategy  dictates  that  funds  available  be 
used  for  features  which  can  be  shown  to  augment 
training  effectiveness. 

Other  recent  studies  by  the  Air  Force  Human 
Resources  Laboratory  (AFHRL)  have  addressed 
various  aspects  of  motion  cueing  (e.g,,  Gray  & 
Fuller,  1577a,  1977b).  It  should  bo  pointed  out. 
however,  that  none  of  these  earlier  studies 
involved  a wide  angle  visual  system  nor  were  the 
tasks  investigated  representative  of  the  more 
complex  flying  tasks.  A number  of  recent 
investigations  at  the  Flying  Training  Division 
(AFHRL/ FT)  using  the  Advanced  Simulator  for 
Pilot  Training  (ASPT)  indicated  that  platform 
motion  did  not  enhance  simulator  training 
effectiveness  for  some  Undergraduate  Pilot 
Training  (UPT)  tasks  (Martin  & Waag,  1978a, 
1978b;  Woodruff,  Smith,  Fuller,  8c  Weyer,  1976). 
The  Tactical  Air  Command  (TAG)  was  extremely 
interested  in  the  implication  of  these  findings  in 
view  of  their  planned  simulator  procurements.  The 
applicability  of  UPT  research  results  to  the  major 
TAC  missions,  i.e.,  air-to-surface  weapons  delivery 
(A/S)  and  air-to-air  combat  (A/ A),  was  questioned. 
It  was  noted  that  not  only  are  TAC  pilots  and 
pilot  students  more  experienced  than  UPT 
students  but  that  they  also  fly  higher  performance 
aircraft  and  perform  more  complex  tasks. 

The  question  at  issue  is  quite  complex.  It  may 
be  postulated  that  several  factors  are  relevant,  i.c.. 
pilot  experience,  type  of  task,  type  of  aircraft, 
capability  of  training  device,  and  scope  of  traimng 


program.  Regarding  pilot  experience,  it  may  be 
argued  that  novice  pilots  need  motion  cueing  to 
help  them  learn  to  do  the  assigned  task.  More 
experienced  pilots  may  somehow  be  able  to 
compensate  for  an  absence  of  such  cues  and  thus 
may  not  need  simulator  platform  motion. 
Alternatively,  novice  pilots  may  be  so  overloaded 
that  they  would  not  use  motion  cuts.  More 
experienced  pilots,  according  to  this  line  of 
reasoning,  may  be  the  ones  who  benefit  most  from 
platform  motion  cues. 

There  is  another  argument  that  may  be  used  in 
support  of  platform  motion.  Perhaps  the  value  of 
such  motion  cuting  is  its  negative  aspects  rather 
than  its  positive  ones.  In  other  words,  learning 
transfer  to  the  aircraft  is  increased  because 
bouncing  around  the  simulator  may  teach  students 
to  ignore  motion  "cues.”  This  is,  in  effect,  stating 
that  platform  motion  makes  for  a “noisy"  and 
thus  more  realistic  learning  environment  in  the 
simulator.  Subjective  opinion  about  the  utility  of 
platform  motion  was  (and  is)  divided  with  both 
proponents  and  opponents  being  equally  vigorous 
in  supporting  their  position.  Therefore,  a series  of 
"motion  sr-.dies"  involving  TAC  pilots  and 
missions  was  requested. 

Planned  TAC  Motion  Studies 

Initially,  a set  of  four  studies  was  planned:  two 
addressing  platform  motion  cueing  for  A/A  tasks 
and  two  addressing  A/S  tasks.  These  included  (a) 
the  current  A'A  study  using  the  simulator  for 
Air-to-Air  Combat  (SAAC);  (b)  an  A'A  study 
using  Northrop  Corporation’s  cantilever  beam 
Large  Amplitude  Simulator  with  Wide  Angle 
Visual  System  (LAS/WAVS);  (c)  an  AS  study 
using  both  SAAC  and  the  model  board  from 
Weapons  System  Training  Set  (WSTS)  #18;  and 
(d)  an  A/S  study  using  AFHRL’s  Advanced 
Simulator  for  Pilot  Training  (ASPT)  at  Williams 
AFB.  Both  the  LAS/WAVS  A/A  study  and  the 
SAAC/WSTS  #18  A/S  study  were  cancelled  at 
TAC’s  request,  but  the  ASPT  A/S  study  was 
completed  (Gray  & Fuller,  1977a). 

Study  Objective 

The  specific  objective  of  this  study  was  to 
determine  the  contribution  of 
six-degrees-of-freedom  platform  motion  to  the 
training  effectiveness  of  the  SAAC  for  initial 
training  of  F-4  air-to-air  combat  skills.  This 
objective  was  met  in  two  steps:  (a)  the  basic 


5 


effectiveness  of  SAAC  training  was  to  be 
established,  and  (b)  the  change  in  effectiveness,  if 
any,  due  to  the  addition  of  platform  motion  was 
to  be  assessed. 


II.  METHOD 

The  study  was  designed  to  be  conducted  in  the 
most  operationally  realistic  manner  possible. 
Departures  from  normal  training  procedures  were 
kept  to  a minimum  in  order  to  enhance  the 
generality  of  the  results. 

Maneuver  Selection 

In  studying  the  effects  of  motion  cueing  on  the 
learning  of  Air  Combat  Maneuvering  (ACM)  skills, 
theoretically,  all  air  combat  maneuvers  in  which 
motion  cueing  is  likely  to  have  an  effect  could  be 
investigated.  There  are,  however,  some  problems 
with  this  concept.  First,  as  maneuvering  becomes 
more  advanced,  measurement  becomes  more  of  a 
problem.  Criteria  are  less  well  defined  and  dif- 
ferent maneuvering  combinations  are  used  to 
achieve  the  same  end.  Second,  training  device 
capability  becomes  a problem.  While  a significant 
portion  of  F-4  air-to-air  training  emphasizes  two 
vs.  one  maneuvering,  the  SAAC  can  only  present  a 
single  aircraft  image  in  the  visual  scene  in  each 
cockpit.  The  SAAC  does  have  the  capability  to  use 
the  spot  it  generates  as  a “sun"  image  for  a 
wingman  “image"  in  two  vs.  one  maneuvering. 
However,  during  die  SAAC  Follow-On  Operational 
Test  and  Evaluation  (FOT&E)  (TAC  Project 
75A-04OU),  this  arrangement  was  found  to  be 
very  unsatisfactory,  particularly  for  students  ex- 
periencing two  vs.  one  maneuvering  for  the  first 
time.  The  third,  and  probably  most  important, 
problem  is  that  the  current  understanding  of  how 
motion  cues  are  perceived  by  pilots  is  not  yet 
sufficient  to  say  motion  cues  are  important  in 
maneuver  A and  not  important  in  maneuver  B. 

Consideration  of  the  above  concepts  led  to  the 
selection  of  the  Basic  Fighter  Maneuvers  (BFM) 
phase  of  training  for  the  transfer-of-training  inves- 
tigation. During  the  early  part  of  this  phase,  pilot 
students  arc  taught  the  classical  maneuvers  used  in 
ACM,  i.c.,  Immelmann  Attack,  Barrel  Roll  Attack, 
High  Yo-Yo,  etc.  Performance  measurement  in  this 
context  is  likely  to  be  more  precise  than  in  later 
stages  of  ACM  training.  The  tasks  are  quite  well 
defined  and  are  practiced  from  fairly  standard 
situations,  and  the  conditions  of  training  for 
different  students  are  more  similar  during  this 
stage  than  for  later  stages  of  training.  The  set  of 
maneuvers  investigated  is  listed  in  Table  1.  Both 
“Tactical  Formation"  and  “Set  up  on  Perch"  are 


Table  1.  BFM  Tadts  (Set  1 Variables) 

Acceleration  Maneuver 
High  Yo-Yo 
Quarter  Plane 
Bartel  Roll  Attack 
Immelmann  Attack 
Lag  Roll 
Separation 
Tactical  Formation 
Set  up  on  Perch 
Defensive  Maneuvers 


not  part  of  the  BFM  but  are  flying  tasks  normally 
practiced  during  the  BFM  training  phase.  All  of 
these  tasks  involve  control  input  behavior;  that  is, 
stick,  throttle,  and  rudder  movement  by  pilots. 
During  BFM  training,  students  are  also  rated  On  a 
set  of  tasks  which  arc  more  cognitive  or  less 
control  in,m'  oriented  than  are  the  classic  BFM 
tasks.  Thu  set  m BFM  “skills"  (Table  2)  was 
included  in  the  study  because  it  was  thought  that 
they  may  be  enhanced  by  simulator  training. 
These  skills  are  practiced  during  the  execution  of 
BFM  tasks. 

Table  2.  BFM  Ski's  (Set  2 Variables) 

Descriptive  Commentary 

Range  Estimation 

Target  Acquisition 

Kept  Bogey  in  Sight 

Weapons  Parameters  Recognition 

Switchology 

Preparation 

Attitude 

Judgment 

Subjects 

The  pilot  students  assigned  to  the  F-4  B-course 
classes  77-EBL  and  77-FBL  at  Luke  AFB  served  as 
subjects  for  this  study.  There  were  22  students,  1 1 
assigned  to  the  311  Tactical  Fighter  Training 
Squadron  (TFTS)  and  11  assigned  to  the  426 
TFTS.  The  1 1 students  in  each  class  were  divided 
into  three  groups  as  shown  in  Tabic  3: 

Table  3.  Subject  Grouping 


Group 

cuts 

77  EBL.  77  FBL 

Total 

Motion 

4 

4 

8 

No-Motion 

4 

4 

8 

No-SAAC 

3 

3 

6 

22 

6 


Groups  were  matched  by  first  ranking  the  pilots  in 
each  class  based  on  their  performance  during  UPT. 
Fighter  Lead-In  Training,  and  progress  to  date  at 
Luke  A FB.  Squadron,  Instructional  System 
IX'velopment  (ISD),  and  AFHRL  personnel  parti- 
cipated in  making  the  rankings.  Group  assignment 
was  then  made  using  a counterbalancing  pro- 
cedure. Two  students  in  each  class  Had  previous 
experience,  as  F-4  Weapon  System  Officer,  and 
these  were  assigned  to  the  Motion  and  No-Motion 
groups  prior  to  assignment  of  the  remainder  of  the 
students. 

Instructors 

Instruction  in  both  the  SAAC  and  the  aircraft 
was  given  by  the  regular  squadron  Instructor  Pilots 
(IPs).  Normal  procedures  for  pairing  students  and 
instructors  were  followed  for  all  sorties  except  the 
SAAC  checkride  (SAAC  7).  All  SAAC  7 missions 
were  given  by  one  of  two  IPs  (one  from  TAWC7 
OLAH  and  one  from  the  TAC  ISD  squadron)  with 
each  IP  assigned  to  rate  half  of  the  pilot  students 
in  the  Motion  and  No-Motion  groups,  respectively. 
Most  of  the  instructor  pilots  had  several  hours  of 
experience  in  the  SAAC,  and  many  had  used  it  to 
train  students  during  the  SAAC  FOT&E;  others 
had  participated  in  continuation  training  sorties  in 
the  SAAC.  The  SAAC-cxpcrienced  instructors 
were  given  a brief  refresher  sortie  in  which  the 
modified  freeze  feature  and  setups  available  were 
demonstrated.  Those  instructors  who  had  not 
flown  previously  were  given  two  familiarization 
sorties  by  either  TAWC/OLAH  or  ISD  personnel. 
All  were  encouraged  to  use  the  freeze,  reset,  and 
other  instructional  capabilities  of  the  SAAC. 

Apparatus 

The  apparatus  used  in  the  study  consisted  of 
the  SAAC  and  the  F-4C  aircraft.  A detailed 
description  of  the  SAAC  is  contained  in  the 
FOT&E  reports  (TAC  Project  75A-04OU),  and 
only  a brief  description  will  be  presented  here. 

The  SAAC  is  comprised  of  two  F-4  cockpits, 
mounted  on  synergistic  six-degrccs-of-frccdom 
motion  systems.  Each  cockpit  is  enclosed  by  a 
matrix  of  eight  pentagonal  cathode  ray  tube 
windows  providing  a 296-  by  150  degree  field  of 
view.  An  electronic  synthetic  terrain  generator  and 
a camera  model  aircraft  image  generator  provide 
the  visual  display.  A g-suit  and  g scat  system 
augments  the  buffet  and  vibration  cues  derived 
from  the  motion  system.  Missile  and  gunfire  tra- 
jectories arc  computed  through  a scoring  system 
that  displays  a simulated  “kill”  during 
engagements. 


Maintenance  on  the  SAAC  is  accomplished  via 
contract  with  the  Simulation  Products  Division  of 
Singer  Company.  Prior  to  the  conduct  of  this 
study.  Singer  reported  that  the  motion  system  on 
SAAC  was  adjusted  to  specifications  and  was 
performing  properly.  During  this  study,  normal 
maintenance  procedures  were  followed.  A problem 
with  one  of  the  legs  was  noted  approximately  2 
weeks  after  completion  of  the  data  collection. 
Examination  revealed  that  failure  of  a component 
in  one  of  the  legs  resulted  in  a small  degree  of 
cross-coupl-ng  in  two  of  the  degrees  of  freedom, 
but  it  was  not  possible  to  determine  when  this 
failure  had  occurred. 

Two  modifications  to  the  system  were  re- 
quested well  prior  to  the  conduct  of  this  study. 
During  the  SAAC  FOT&E,  it  was  found  that 
instructors  were  reluctant  to  use  the  SAAC  in- 
structional features  other  than  reset.  Due  to  the 
low  reliability,  or  operating  inconvenience  of  these 
features,  AFHRL  personnel  suggested  that  gieatcr 
use  of  the  problem  freeze  and  rccord/playback 
features,  in  particular,  might  have  a strong  poten- 
tial for  increasing  student  learning.  The  problem 
freeze  allows  :he  instructor  to  stop  the  two  cock- 
pits at  any  point.  The  capability  to  stop  and  give 
the  student  feedback  about  what  has  just  occurred 
can  be  a powerful  instructional  tool.  Prior  to  the 
current  study,  the  use  of  the  freeze  feature  in 
SAAC  required  the  instructor,  who  was  normally 
in  one  coekpit,  to  request  the  console  operator  to 
initiate  a freeze.  Even  when  the  operator  re- 
sponded immediately,  the  delay  due  to  the  time 
reuuircd  for  communication  often  resulted  in  the 
desired  situation  being  lost.  Prior  to  the  current 
study,  the  nose  wheel  steering  switch  on  the 
control  stick  was  wired  to  become  a freeze 
control.  A freeze  could  then  be  initiated  or 
released  from  either  cockpit. 

Similarly,  the  rccord/playback  feature  had  not 
worked  reliably  on  SAAC.  In  addition,  the  series 
of  operations  required  of  the  console  operator  to 
use  the  feature  were  complex  enough  to  result  in 
frequent  errors.  As  a result,  record/ playback  was 
rarely  used.  In  light  of  this,  modifications  were 
requested  to  improve  system  reliability  and  ease  of 
operation  and  to  provide  a "visual  slaving" 
capability  in  the  playback  mode.  This  would  allow 
both  cockpits  (i.e.,  instructor  and  student)  to  view 
a just  recorded  engagement  from  either  the  stu- 
dent's  or  the  instructor's  viewpoint.  It  was  thought 
that  this  capability  could  be  a valuable  feedback 
tool,  particularly  for  students  who  arc  being 
taught  ACM  fundamentals  for  the  first  time. 
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Technological  problems  resulted  in  these  modifi- 
cations not  being  completed  in  time  to  support  the 
study. 

Training  Syllabus 

The  normal  training  syllabus  was  followed 
except  for  modifications  necessary  to  support  the 
study.  These  exceptions  involved  moving  forward 
some  ol  the  relevant  ACM  academics,  adding  seven 
SAAC  simulator  sorties  for  the  Motion  and  No- 
Motion  groups,  and  adding  an  additional  BFM 
sortie  in  the  aircraft.  The  SAAC  portion  of  the 
syllabus  consisted  of  two  familiarization  sorties, 
four  training  soritics,  and  a checkride  sortie  The 
additional  aircraft  BFM  sortie  was  inserted  to  in- 
crease the  number  of  practice  trials  for  those 
maneuvers  under  investigation.  The  relevant 
portions  of  the  training  syllabus  are  shown  at 
Appendix  A. 

A "blocked”  training  model  was  followed  in 
building  the  syllabus,  in  that  all  SAAC  training  was 
scheduled  to  be  completed  prior  to  beginning  the 
corresponding  aircraft  sorties.  This  mode  of 
training  was  chosen  to  more  effectively  demon- 
strate any  training  effects  of  the  SAAC  and  to 
facilitate  scheduling. 

The  number  of  SAAC  sorties  included  was 
based  on  experience  with  the  SAAC  FOT&E.  Two 
short  orientation  sorties  rather  than  a single  longer 
one  was  thought  to  be  more  appropriate,  partic- 
ularly for  those  few  students  who  may  experience 
a first  time  queasiness  in  an  advanced  simulator 
such  as  SAAC.  It  was  also  noted  during  the  SAAC 
FOT&E  that  four  to  five  BFM  training  sorties 
tended  to  bring  students  to  a plateau  where  they 
were  anxious  to  try  BFM  in  the  aircraft  and  were 
just  beginning  to  lose  motivation  for  repeatedly 
performing  BFM  tasks  in  the  SAAC. 

Instruction  in  the  SAAC 

During  the  simulator  training,  an  IP  flew  one 
cockpit  and  the  student  flew  the  other.  All 
students  flew  the  same  cockpit,  displaying  an  F-4 
aircraft  image  in  its  visual  scene.  It  was  decided 
that  such  a mode  of  operation  was  the  best  avail- 
able option  for  this  study.  Instructing  the  students 
from  the  console  was  not  attempted  because  of 
the  limited  visual  feedback  available  to  the  in- 
structor. While  the  current  instructional  capa- 
bilities may  have  been  enhanced  by  having  two  IPs 
present,  one  at  the  console  and  the  other  in  one 
cockpit,  manning  levels  available  at  the  time  were 
not  sufficient  to  support  this  option.  The  console 
was  operated  by  AFHRL  personnel.  Ail  instruc- 
tional decisions  were  made  by  the  IP.  When  asked, 


AFHRL  research  personnel  provided  feedback 
concerning  such  information  as  range  and  aspect 
angle  from  displays  on  the  console. 

System  Configuration 

Both  the  g-seat  and  the  g-suit  were  used  by  the 
instructors  and  students  for  all  simulator  training 
sessions.  The  use  of  platform  motion  for  subjects 
was  determined  by  group  assignment.  The  Morion 
group  students  flew  SAAC  with  platform  motion 
for  all  training;  the  No-Motion  group  students  flew 
SAAC  without  platform  motion  for  all  training. 
The  instructors  had  the  option  of  flying  SAAC 
with  or  without  platform  motion,  but  most 
elected  not  to  use  it. 

Grading  Scale 

TAC  nominally  uses  a 5-point  grading  scale 
(0-4).  The  definitions  of  these  grades  as  printed  on 
the  gyade  sheet  are  given  in  Appendix  B.  In 
practice,  IPs  report  that  the  large  majority  of 
grades  actually  given  are  l's  and  2’s.  A few  grades 
of  0 and  3 are  given  and  grades  of  4 , although  used 
very  rarely  for  items  like  "Attitude"  or  "Prepara- 
tion,” arc  essentially  nonexistent.  It  was  thought 
that  the  normal  grading  system  was  inadequate  for 
the  present  study  because,  for  most  maneuvering, 
it  was  essentially  a 2-point  scale  (1  & 2).  In  order 
to  provide  more  precision  in  measuring  the  effects 
of  simulator  training  and  differences  due  to 
motion,  a more  precise  grading  scale  was  neces- 
sary. A variety  of  scales  was  considered  in 
discussions  with  1SD  and  instructor  personnel.  It 
was  decided  that  the  current  scale  could  be  ex- 
panded to  increase  precision  by  adding  “plus”  and 
"minus"  categories.  Such  a procedure  did  not 
require  extensive  learning  of  new  criteria  or  anchor 
points  by  the  instructors. 

Grading  Instructions 

Squadron  personnel  were  briefed  on  the  special 
grading  procedures  to  be  used  for  the  study,  and  a 
grading  guide  was  left  at  the  squadron  for  use 
during  the  four  specially  graded  aircraft  sorties. 
Instructors  were  asked  to  rate  each  individual 
maneuver  on  a special  grade  card  both  in  the 
simulator  and  in  the  aircraft.  These  grades  were  in 
addition  to  the  normal  method  of  a single  grade 
for  each  type  of  maneuver.  Thus,  if  a student  did 
three  acceleration  maneuvers  on  a given  day.  four 
ratings  for  that  maneuver  were  made:  a raring  for 
each  of  the  three  “trials”  and  an  overall  or  “daily” 
grade  for  that  task. 

Data  Collected 

The  primary  data  consisted  of  the  performance 
ratings  made  during  the  four  aircraft  sorties,  BFM 
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4 through  BFM  6B.  During  SAAC  training,  rating 
data  were  collected  for  each  of  the  maneuvers 
listed  in  Table  1 and  for  each  of  the  skills  listed  in 
Table  2.  In  addition,  questionnaire  data  were  col- 
lected from  both  students  and  instructors.  The 
questions  asked  were  of  a non-directive  nature 
(Appendix  C).  It  should  be  noted  that  most  of  the 
instructors  involved  had  recently  completed  very 
detailed  questionnaires  during  the  SAAC  FOT&E, 
the  results  of  which  art  to  be  described  in  a Final 
Report  by  TAC  (TAC  Project  75A-04OU). 

III.  DATA  ANALYSES  AND  RESULTS 

For  the  purposes  of  analysis,  the  performance 
ratings  made  by  the  IPs  were  first  rescaled  ac- 
cording to  Table  4.  Several  one-way  analyses  of 
variance  (ANOVAs)  (Evanson,  1975)  were  per- 
formed to  assess  the  significance  of  the  differences 
in  performance  among  the  three  groups.  It  was 
decided  that  the  normal  p < .05  and  p < .01 
criteria  for  significance  were  too  strict  for  the 
current  study;  therefore,  a more  liberal  p < .10 
was  used.  With  this  criterion  a difference  among 
groups  is  termed  significant  when  it  may  be  ex- 
pected to  occur  by  chance  no  more  than  one  time 
in  ten.  The  choice  of  this  liberal  criterion  meant 
that  there  was  more  of  a chance  that  increased 
performance  increments  due  to  motion  were 
labelled  significant. 


Table  4.  Transformation  Used 
Prior  to  Analyses 


Rating 

Qlvon 

Rating  Usod 
tor  Analysis 

3+ 

9 

3 

8 

3- 

7 

2+ 

6 

2 

5 

2- 

4 

1+ 

3 

1 

2 

1- 

1 

0 

0 

Each  analysis  was  accomplished  to  answer  a 
specific  question.  These  questions  are  addressed 
separately  in  the  following  sections  with  both  the 
analysis  performed  and  the  results  obtained 
described.  The  simulator  data  arc  presented  bsforc 
aircraft  data.  Table  5 lists  the  mean  ratings  given 
for  each  group  at  different  points  in  training;  each 


Table  5.  Mean  Performance  Ratings  for 
BFM  Maneuvers 


Group 

Training  mm 

Motion 

Initial  Simulator 

3.94 

3.93 

Final  Simulator 

— 

4.85 

5.45 

Initial  Aircraft 

4.03 

3.73 

3.64 

Final  Aircraft 

4.89 

4.56 

4.47 

Overall  Aircraft 

4.44 

4.20 

4.13 

entry  is  averaged 

across  s 

iubjects  and  the 

BFM 

maneuvers  listed  in  Table  1 

. Table  6 similarly  lists 

the  mean  ratings 

for  each 

group  averaged 

across 

subjects  and  the  BFM  skills  listed  in  Table  2. 

Table  6.  Mean  Performance  Ratings  for  BFM  Skills 


OrpHp 


Training  PtilM 

NO-SAAC 

No -Motion 

Motion 

Initial  Simulator 

_ 

5.65 

5.02 

Final  Simulator 

_ 

5.58 

6.49 

Initial  Aircraft 

5.17 

4.79 

4.96 

Final  Aircraft 

5.22 

5.24 

5.19 

Overall  Aircraft 

4.98 

5.12 

5.12 

Initial  Group  Equality 

The  first  question  of  interest  concerns  the 
relative  performance  of  the  Motion  and  No-Motion 
groups  at  the  beginning  of  simulator  training.  It 
was  stated  earlier  that  groups  were  matched  to  the 
greatest  possible  extent.  If  the  matching  was  suc- 
cessful, consistent  differences  in  performance 
between  the  Motion  and  No-Motion  groups  would 
not  be  expected.  To  address  this  issue,  the  first 
daily  ratings  for  each  maneuver  and  skill  were 
analyzed.  These  ratings  were  generally  from  simu- 
lator sorties  3 or  4,  sorties  1 and  2 being  dedicated 
to  simulator  orientation.  Two  kinds  of  analyses 
were  made.  In  one  type  of  analysis,  performance 
on  each  maneuver  was  considered  in  a separate 
ANOVA.  For  purposes  of  discussion,  this  type  of 
analysis  will  be  termed  a “secondary”  analysis. 
Using  this  method,  none  of  the  differences  was 
significant  for  any  of  the  Set  1 or  Set  2 Variables. 
The  mean  ratings  for  each  group  and  task  are 
shown  in  Table  D1  in  Appendix  D.  Also  presented 
is  the  significance  probability  from  the  ANOVA 
for  each  maneuver.  It  should  be  noted  that  when 
large  numbers  of  ANOVAs  are  performed,  as  was 
done  in  the  secondary  analysis,  some  “significant” 
differences  are  likely  to  be  due  to  chance  'i.e. 
approximately  one  in  ten  for  the  chosen  criterion). 
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However,  if  a large  i umber  ot  significant  dif- 
ferences axe  found,  it  may  be  assumed  with 
confidence  that  real  differences  do  exist. 

Since  considering  each  maneuver  or  skill 
separately  does  not  give  a picture  of  overall  ef- 
fects, another  type  of  analysis  was  conducted.  This 
analysis  grouped  together  the  BFM  tasks  (Set  1 
Variables)  as  a set  of  repeated  measures.  The  BFM 
skills  were  similarly  grouped  together  as  a second 
set  of  repeated  measures.  This  was  done  under  the 
assumption  that,  although  the  differences  for  the 
particular  maneuvers  or  skills  may  be  small,  they 
may  be  important  if  one  group  consistently 
performs  better  than  the  other  group  at  most  or  all 
tasks.  In  the  following  paragraphs,  this  type  of 
analysis  will  be  referred  to  as  a “primary”  analysis 
because  it  considers  tasks  collectively.  For  Set  1 
Variables,  the  difference  between  the  two  groups 
was  not  significant,  but  for  Set  2 Variables,  the 
No-Motion  group  had  significantly  higher  ratings. 
The  mean  performance  ratings  for  both  groups  are 
shown  in  the  “Initial  Simulator"  row  in  Tables  5 
and  6.  The  means  and  significance  levels  from  the 
primary  analysis  are  also  included  in  Table  Dl. 

Learning  in  the  Simulator 

The  second  question  of  interest  is  whether 
students  learned  in  the  simulator.  If  learning  had 
occurred,  higher  ratings  during  the  last  sortie 
would  be  expected.  To  address  this  question,  the 
fust  daily  rating  for  each  variable  for  each  group 
was  compared  to  ratings  given  during  the  last  simu- 
lator sortie,  SAAC  7.  A primary  analysis  con- 
firmed these  expectations.  Highly  significant 
differences  were  noted  for  both  sets  of  variables 
for  the  Motion  group  and  for  the  Set  1 Variables 
for  the  No-Motion  group.  Differences  on  Set  2 
Variables  were  not  significant  for  the  No-Motion 
group.  In  fact,  the  mean  initial  and  final  perform- 
ance ratings  on  Set  2 Variables  were  almost 
identical. 

When  Fust  vs.  last  ratings  were  compared  for 
each  task  through  a secondary  analysis,  differences 
were  consistently  in  the  expected  direction  for  all 
but  the  No-Motion  Set  2 Variables.  Significant 
differences  were  also  found  in  the  Set  1 Variables. 
The  significant  probabilities  for  these  first  vs.  last 
ANOVAs  are  listed  in  Table  D6. 

Final  Simulator  Standings 

The  remaining  question  of  interest  concerns  the 
status  of  the  two  groups  as  they  completed  simu- 
lator training.  A primary  analysis  compared  the 
performance  of  both  groups  for  each  set  of 


variables.  For  both  sets  of  variables,  the  ratings 
received  by  the  Motion  group  were  significantly 
better  than  those  of  the  No-Motion  group.  There 
was,  however,  a problem.  It  was  intended  that 
each  of  the  two  IPs  who  conducted  the  SAAC  7 
chcckridcs  should  evaluate  four  students  from 
each  group.  A scheduling  error  resulted  in  one  IP 
evaluating  five  No-Motion  and  three  Motion 
students  and  the  other  IP  evaluating  three  No- 
Motion  and  five  Motion  students.  The  result  of 
this  partial  confounding  of  group  and  IP  evaluator 
is  that  the  higher  ratings  of  the  motion  group  may 
be  due  to  IP  bias,  to  motion,  or  to  both.  The 
means  and  significance  levels  for  these  data  are 
shown  in  Table  D2. 

Analyses  of  Aircraft  Data 

The  data  were  analyzed  to  measure  transfer  of 
training  to  the  aircraft.  The  analyses  were  similar 
to  those  used  lor  the  simulator  data.  In  addition  to 
the  data  analyses  of  initial  performance  levels, 
final  performance  levels,  and  learning  effects,  an 
overall  category  was  included  which  averaged 
together  all  daily  ratings  for  each  task  in  the  air- 
craft for  each  subject.  The  same  types  of  analyses 
were  conducted  on  both  trial  and  daily  rating  data 
for  Set  1 Variables.  Since  the  results  from  the 
analysis  of  trial  data  are  very  similar  to  those  ob- 
tained with  the  daily  rating  data,  only  the  latter 
are  presented.  As  with  the  simulator  data,  the 
means  and  significance  probabilities  from  the 
ANOVAs  are  presented  in  Appendix  D.  The  means 
from  the  primary  analysis  are  also  shown  in  Tables 
5 and  6. 

initial  Aircraft  Data 

There  are  several  hypotheses  about  how  and 
where  the  effectiveness  of  simulator  trained  groups 
should  show  up.  The  most  common  of  these  is 
that  simulator  trained  groups  should  exhibit  a 
higher  "initial"  skill  level  when  the  aircraft  phase 
of  training  is  begun.  When  simulator  training  is 
“blocked,”  that,  completed  prior  to  aircraft 
training  rather  than  interspersed  with  it,  tliis 
expectation  seems  quite  reasonable.  A primary 
analysis  of  the  first  daily  ratings  for  each  task 
resulted  in  differences  among  the  three  groups  that 
were  not  significant.  However,  and  somewhat  sur- 
prisingly, the  mean  rating  for  the  No-SAAC  group 
was  slightly  higher  than  those  for  the  two  SAAC 
trained  groups,  which  differed  only  slightly. 

When  the  data  for  the  Motion  and  No-Motion 
groups  are  combined,  the  analysis  becomes  a 
comparison  of  SAAC  trained  vs.  No-SAAC  trained 


subjects.  Even  though  such  a test  is  more  powerful 
due  to  increased  sample  size,  the  SAAC  vs. 
No-SAAC  difference  dees  not  reach  significance 
for  either  set  of  variables. 

When  the  initial  aircraft  data  for  the  three 
groups  arc  analyzed  for  each  task  separately,  a 
secondary  analysis,  two  of  the  differences  were 
significant  but  were  not  in  a consistent  direction 
and  arc  no  more  than  would  be  expected  by 
chance. 

Learning  in  the  Aircraft 

The  next  analysis  was  concerned  with  whether 
the  ratings  showed  that  learning  occurred  in  the 
aircraft.  It  is  not  obvious  that  ratings  necessarily 
reflect  the  degree  of  learning  taking  place.  Current 
economics  do  not  allow  the  same  maneuver  to  be 
continually  practiced  under  the  same  conditions. 
As  soon  as  a student  attains  some  degree  of 
proficiency,  the  conditions  of  practice  for  a parti- 
cular maneuver  may  change  just  enough  that  the 
student  remains  challenged  but  still  experiences 
some  success.  This  is  particularly  true  in  the  air 
combat  phases  of  training.  For  example,  a 
student’s  first  attempts  at  Acceleration  Maneuvers 
and  Hi  Yo-Yos  arc  likely  to  occur  in  very 
standardized  situations.  However,  the  last  trials 
occurring  in  the  sorties  under  study  may  be  in 
conditions  which  more  nearly  resemble  free 
engagement.  The  effect  of  these  changing  condi- 
tions on  IP  grading  criteria  and  resultant  ratings 
arc  unknown.  However,  it  is  reasonable  to  assume 
that  these  same  unknown  factors  affect  all  three 
groups. 

The  first  daily  rating  vs.  last  daily  rating  com- 
parison was  made  for  each  group  separately  for 
both  Set  1 and  Set  2 Variables.  The  results  of  the 
primary  analyses  for  all  three  groups  were  the 
same.  Oi  Set  1 Variables,  the  last  daily  ratings  were 
significantly  higher  than  the  early  ratings.  On  Set  2 
Variables,  the  differences  were  in  the  expected 
direction  but  did  not  reach  significance.  When  the 
data  from  all  three  groups  are  combined,  the 
difference  reaches  significance  (p=.08). 

As  with  the  simulator  data,  a secondary  analysis 
resulted  in  more  significant  differences  for  Set  1 
Variables.  The  significance  probabilities  from  these 
ANOVAs  arc  also  listed  in  Table  D6. 

Final  Aircraft  Data 

In  a previous  section,  the  expectation  that 
simulator  trained  groups  may  enter  the  aircraft 
phase  of  training  with  an  advantage  was  discussed 
and  basically  rejected.  The  groups  were  not  found 


to  be  significantly  different.  Several  hypotheses 
may  be  generated  concerning  the  comparative 
state  of  the  groups  at  the  end  of  the  period  under 
investigation.  There  may  be  no  differences,  which 
implies  either  that  they  never  existed  or  that  any 
that  did  exist  have  been  obscured.  SAAC  trained 
groups  may  rate  higher  if  the  basic  simulator  ef- 
fectiveness shows  up  later  in  training  rather  than 
initially.  This  result  can  occur  if  the  SAAC  trained 
students  have  acquired  what  is  termed  a better 
“situation  awareness”  which  could  exhibit  itself 
after  an  initial  settling  period.  On  the  other  hand, 
if  the  SAAC  trained  students  have  picked  up  some 
bad  habits,  they  may  not  perform  as  well. 

A primary  analysis  of  the  last  daily  ratings 
indicated  a significant  difference  for  the  Set  1 
Variables  collectively.  Surprisingly,  the  difference 
was  in  favor  of  the  No-SAAC  group.  Differences 
between  the  two  SAAC  trained  groups  appeared  to 
be  negligible.  The  differences  for  Set  2 Variables 
were  not  significant. 

Overall  Aircraft  Data 

For  this  analysis,  all  of  the  daily  rating  data 
were  used.  For  all  variables,  the  ratings  were 
averaged  together  for  each  task  and  subject.  For 
the  Set  1 Variables,  the  differences  again  were  in 
favor  of  the  No-SAAC  group  and  approach  signifi- 
cance (p=.12).  For  Set  2 Variables,  the  difference 
between  groups  appeared  to  be  negligible.  On  the 
secondary  analysis  the  only  significant  effect  was 
fo:  the  acceleration  maneuver.  The  No-SAAC 
group  again  scored  highest. 

Questionnaire  Data 

Since  the  questionnaire  data  were  not  quantita- 
tive, they  were  not  submitted  to  a statistical 
analysis.  Students  responded  that  SAAC  training 
helped  them  ‘o  acquire  an  initial  advantage  in 
being  better  able  to  picture  BFM  maneuvers.  There 
were  mixed  comments  aj  to  whether  the  SAAC 
training  actually  helped  them  to  perform  the 
maneuvers  any  better.  Student  opinion  with  regard 
to  platform  motion  generally  indicated  that  they 
thought  it  did  not  or  would  do  nothing  to  help 
them.  Those  students  that  flew  with  platform 
motion  often  stated  that  after  the  first  two  or 
three  sorties  they  did  not  pay  any  attention  to  it. 

Instructor  opinions  were  generally  in  agreement 
in  that  they  perceived  no  differences  between  the 
SAAC  and  No-SAAC  trained  students  insofar  as 
performance  in  the  aircraft  was  concerned.  A small 
number  of  the  IP  thought  that  SAAC  trained 
students  did  have  a slight  tni.ial  advantage  in 


11 


performance.  It  was  thought  that  this  advantage 
disappeared  after  one  or  two  sorties.  Several  of  the 
IPs  also  agreed  with  the  students  that  SAAC  gave 
the  students  a good  initial  “picture”  of  BFM  tasks. 
IP  comments  about  motion  were  misted,  with 
those  not  in  favor  of  motion  making  the  stronger 
statements.  As  mentioned  previously,  most  of  the 
IPs  when  given  the  choice  during  the  training  of 
students  opted  not  to  use  platform  motion  for 
themselves. 

The  Tactical  Air  Command  SAAC  FOTfitE  final 
report  and  the  Simulator  Comparative  Evaluation 
report  contain  a more  comprehensive  discussion  of 
questionnaire  data  about  the  SAAC. 

IV.  DISCUSSION  AND  RECOMMENDATIONS 

The  objective  of  this  study  was  to  determine 
the  contribution  of  six-degrees-of-freedom  plat- 
form motion  to  the  training  effectiveness  of  the 
SAAC  for  initial  training  in  A/ A tasks.  There  were 
two  primary  parts  to  this  objective:  (a)  the  basic 
training  effectiveness  of  the  SAAC  and  (b)  the 
increment  in  effectiveness  when  platform  motion 
is  used.  The  current  data  did  not  reflect  any 
noticeable  increment  of  effectiveness  due  to  plat- 
form motion.  There  were  no  consistent  transfe,- 
of-training  differences  between  the  Motion  and 
No-Motion  groups.  This  result,  however,  must  be 
interpreted  in  the  context  of  a lack  of  demon- 
strated training  effectiveness  of  the  SAAC  itself. 
The  data  indicated  that  SAAC  training  did  not 
seem  to  increase  instructor  ratings  of  performance 
in  the  aircraft.  If  anything,  there  is  some  indica- 
tion that  the  students  who  did  not  receive  SAAC 
training  performed  better  in  the  aircraft.  This  basic 
non-effectiveness  will  be  discussed  at  length  prior 
to  returning  to  the  question  of  motion.  The  reader 
should  keep  in  mind  the  integral  relationship  of 
training  program  and  training  device.  Training  ef- 
fectiveness is  a function  of  both  of  these.  Even 
though  significant  training  effectiveness  was  not 
demonstrated  under  the  conditions  and  re- 
strictions of  the  current  study,  this  does  not  mean 
that  all  SAAC  training  will  be  noneffective.  It  only 
means  that  the  right  training  program  and  training 
device  capability  combination  was  not 
demonstrated  in  this  study. 

Relationship  to  Previous  Results 

To  date,  there  have  been  only  two  studies  that 
are  closely  related  to  the  current  study.  The  first 
of  these  (Payne,  Hirsch,  Sc  Temple,  1976)  involved 
the  training  of  Navy  F-4  pilots  on  Northrop's  air 
combat  simulator.  That  study  was  a very 


structured  training  experiment  involving  extensive 
experimental  control  including  fixed  IP  and 
student  pairings.  In  addition,  up  to  12  perform- 
ance ratings  were  made  for  each  trial  of  each 
maneuver.  The  current  study,  in  an  attempt  to  be 
as  operationally  oriented  as  possible,  imposed  a 
minimum  of  departures  from  normal  training  pro- 
cedures. The  Northrop  study  found  a consistent 
positive  transfer-of-training  effect.  Simulator 
trained  students  for  the  Northrop  study  performed 
better  in  the  aircraft  than  those  students  who  did 
not  receive  the  simulator  training.  Although 
transfer  of  training  was  noted  in  a variety  of  ways, 
differences  were  ,iot  large  enough  to  be  significant 
except  for  the  Rolling  Sdssors  maneuver.  No 
across  maneuvers  or  summary  analyses  were 
reported. 

The  other  study  which  is  most  relevant  was  the 
Phase  II  or  student  training  phase  of  the  SAAC 
FOT&E.  A comparison  of  instructor  ratings  from 
BFM  training  yielded  results  similar  to  those  from 
the  Northrop  study,  a very  small  positive  effect. 
Differences  were  not  significant  in  a statistical 
sense. 

The  most  closely  related  study  done  by 
AFHRL  involved  the  training  of  UPT  students  on 
the  ASPT  in  aerobatic  maneuvers  (Martin  Sc  Waag, 
1978b).  That  study  also  did  not  find  large  transfer 
of  training  effects.  The  only  maneuver  which 
showed  a significant  transfer  of  training  effect  was 
the  Ban-el  Roll. 

The  notable  exception  to  the  small  effects 
noted  above  is  the  A/S  motion  study  (Gray  Sc 
Fuller,  1977a,  1977b).  This  study  found  a very 
significant  transfer  of  training  effect.  The  addition 
of  platform  motion,  however,  did  not  increase  this 
transfer. 

In  summary,  except  for  the  A/S  motion  study, 
the  results  of  the  studies  to  date  that  are  most 
closely  related  to  the  current  study  are  similar  in 
that  the  measurable  effects  of  simulator  training 
for  ACM  related  tasks  tend  to  be  small. 

The  Measurement  Problem 

There  are  several  possible  explanations  for  the 
lack  of  a demonstrable  positive  transfer  of 
training.  One  problem  is  that  of  performance 
measurement.  Performance  measurement  in  the 
aerial  combat  arena  is  a difficult  task.  Standards  of 
performance  are  less  well  defined  than  for  other 
flying  tasks.  For  the  current  study,  it  was  assumed 
that  instructor  ratings  on  the  expanded  scale  used 
were  sufficiently  sensitive  and  reliable.  Even 
though  this  was  the  chosen  option,  this 
assumption  may  be  questioned. 
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Another  possible  problem  with  measurement  is 
more  conceptual.  If  what  a student  learns  in  a 
training  device  like  SAAC  is  an  overall  “situation 
awareness"  as  some  instructors  report,  then  how 
this  knowledge  would  be  exhibited  is  not  precisely 
clear.  It  was  expected  that  this  might  show  up  in 
the  ratings  on  the  Set  2 Variables  since  those  skills 
are  more  cognitive  in  nature  than  the  Set  1 
Variables.  However,  ratings  on  these  variables  were 
less  sensitive  to  training  effects  than  ratings  on  Set 
1 Variables. 

The  remaining  point  about  performance 
measurement  concerns  training  program  modifi- 
cations. Inclusion  of  training  on  a device  like  the 
SAAC  is  not  routinely  justifiable  without 
measurable  transfer  of  training.  This  reservation, 
of  course,  should  not  prevent  experimental  mani- 
pulations in  attempts  to  find  improved  training 
procedures  and  measurement  tools. 

Negative  Training 

It  has  long  been  a well  established  fact  that 
prior  training  in  a simulator  improves  performance 
and/or  learning  in  the  aircraft.  The  degree  of  this 
facilitation  is  a complex  function  of  the  task  to  be 
performed,  training  device  capability,  instructor 
competence,  and  content  and  sequence  of  instruc- 
tion within  the  training  program.  While  opera- 
tional personnel  have  often  expressed  a concern 
that  negative  training  may  occur,  particularly 
when  the  simulator  vs.  aircraft  procedures  or 
handling  characteristics  are  quite  different,  no 
such  negative  effects  have  been  reported  to  date. 
For  this  reason,  the  advantage  of  the  No-SAAC 
group  at  the  end  of  the  training  period  under 
consideration  is  worthy  of  note,  This  performance 
difference  may,  of  course,  be  due  to  a sampling 
bias.  In  spite  of  the  matching  procedures  em- 
ployed, the  better  pilots  may  have  ended  up  in  the 
No-SAAC  group.  Another  potential  explanation 
appeals  to  the  motivation  of  the  individuals  in- 
volved. In  one  of  the  squadrons,  the  No-SAAC 
students  were  reported  to  have  established  a 
rapport  and  a “we’re  good  enough  not  to  need  the 
SAAC  attitude  or  team  spirit.  A third  possibility 
is  that  of  instructor  bias.  The  expectations  of 
instructors  may  have  been  different  for  SAAC 
trained  and  No-SAAC  trained  students.  While 
these  possibilities  cannot  be  discounted,  it  is 
thought  they  should  not  be  relied  oil  either. 
Therefore,  the  SAAC  trai'  ng  program  was 
examined  to  look  for  other  potential  explanations. 
Two  particular  problems  were  noted. 

One  problem  concerns  the  scheduling  of  SAAC 
training.  For  those  students  who  received  it,  SAAC 


training  was  an  additional  activity.  It  is  possible 
that  SAAC  training  interfered  with  other  training 
merely  because  it  took  time  and  required  attention 
at  the  critical  time  when  students  were  just 
learning  basic  aircraft  control  during  transition  and 
formation  training.  The  SAAC  trained  students 
may  have  fallen  increaaingty  farther  behind  their 
classmates.  While  the  difference  from  the  No- 
SAAC  group  was  small  at  the  beginning  of  aircraft 
training,  it  became  large  enough  to  become  signifi- 
cant at  the  end  of  the  period  under  consideration. 
Although  this  hypothesis  is  consistent  with  the 
observed  data,  there  is  no  way  to  verify  that  it 
actually  occurred.  The  im plication  for  training 
program  developers  and  researchers  is  that  too 
many  activities  should  not  be  compering  for  a 
student's  time  and  attention 

The  second  problem  concerned  the  way  in 
which  the  SAAC  training  was  actually  conducted. 
It  is  very  possible  that  the  amount  of  feedback 
which  the  student  received  was  not  adequate  for 
his  current  stage  of  training.  In  this  study,  the  IP 
instructed  from  the  target  aircraft.  While  this  was 
the  best  option  available  in  SAAC  at  the  time,  this 
position  is  certainly  not  optimal.  It  may  not  have 
allowed  the  IP  to  perceive,  and  correct,  the 
development  of  some  minor  “bad  habits”  which 
later  interfered  with  the  student’s  progress.  Note 
that  the  term  “minor"  is  used.  It  is  expected  that 
major  problems  (a)  would  have  been  detectable  by 
the  IP  fiom  his  position  in  the  target  aircraft  in 
SAAC  and  (b)  may  have  resulted  in  obviously 
poorer  performance  in  the  early  aircraft  data.  The 
latter  was  not  reflected  in  the  ratings  oi  mentioned 
by  instructors  on  the  questionnaires. 

A similar  problem  was  the  difficulty  of  ade- 
quately demonstrating  a maneuver  in  the  SAAC. 
Since  the  IP  and  the  student  were  not  able  to  see 
the  same  image  simultaneously,  it  was  hard  to  talk 
the  student  through  a demonstration.  It  was 
because  of  these  difficulties  that  the  "visual 
slaving"  capability  of  the  record/playback  system 
was  requested  earlier. 

Experience,  Instructional  Features, 
and  Effectiveness 

In  the  previous  section  it  was  speculated  that 
the  training  effectiveness  of  the  SAAC  for  initial 
fighter  training  may  be  hampered  by  preventing 
adequate  feedback  by  instructors.  This  need  for 
feedback  is  likely  to  be  greatest  when  students  are 
initially  exposed  to  the  fundamentals  of  air 
combat.  This  was  precisely  the  phase  of  training 
with  which  the  current  studv  dealt.  In  contrast, 
consider  the  situation  with  the  more  experienced 


pilot.  He  is  much  less  likely  to  require  the  same 
kind  of  feedback  from  his  instructor.  He  has 
enough  knowledge  about  air  combat  maneuvering 
that  he  can  monitor  his  own  performance.  He  is 
likely  to  have  a sufficient  concept  of  the  geometry 
of  an  engagement  that  postmission  discussion 
during  debriefing  may  provide  adequate  feedback. 
The  logic  above  may  explain  one  of  the  paradoxes 
in  the  SAAC  FOT&E  results.  The  majority  of 
instructors  felt  that  SAAC  provided  effective 
training  to  them  (the  instructors)  but  was  of 
doubtful  value  for  training  students.  At  the  time, 
this  was  perceived  as  a bias  against  simulation,  but 
perhaps  the  feedback  problem  is  a more  plausible 
explanation. 

The  effectiveness  of  SAAC  for  experienced 
pilots  was  supported  by  questionnaire  data  from 
the  Tactical  Fighter  Weapons  Center  (TFWC)  TAC 
ACES  II  project.  A similar  program  being  con- 
ducted at  Vought  Corporation  in  Dallas,  Texas, 
also  reported  the  effectiveness  of  air  combat 
simulation  for  experienced  pilots  (TAC  Project 
74T-912F). 

Both  the  Vought  and  the  Northrop  air  combat 
simulators  use  a “jump  seat"  arrangement  where 
the  instructor  sits  to  the  side  and  slightly  behind 
the  student.  Both  of  these  devices  also  have  a 
record/playback  capability.  It  is  felt  that  for  initial 
fighter  training,  these  capabilities  would  provide  a 
better  training  potential  than  the  SAAC  arrange- 
ment as  existed  at  the  time  of  this  study.  The 
visual  slaving  previously  discussed  plus  the  capa- 
bility of  maneuvering  against  a pre-recorded  target, 
freeing  the  IP  to  monitor  the  student,  are  seen  as 
the  two  most  promising  enhancements  which  may 
be  made  to  SAAC. 

Simulation  Training  Procedures 

Before  returning  to  the  motion  question,  a few 
additional  observations  about  training  procedures 
will  be  presented.  These  comments  relate  to  the 
use  of  simulator  unique  features.  While  strong 
credit  must  be  given  to  the  instructors,  who 
adapted  fairly  well  to  the  requirement  to  instruct 
using  SAAC,  it  became  obvious  that  they  were 
reluctant  to  use  SAAC  much  differently  than  they 
might  employ  an  aircraft,  that  is,  they  tended  to 
ignore  many  of  the  unique  instructional  features 
of  SAAC.  For  example,  although  the  reset  feature 
was  used  extensively  at  the  completion  of  a 
maneuver,  rarely  was  it  used  to  terminate  a situa- 
tion close  to  the  point  in  time  that  an  error  was 
committed.  Error  situations  were  ususlly  allowed 
to  continue  and  resulted  in  the  compounding  or 
errors.  In  an  aircraft,  this  compounding  of  errors 


would  net  often  happen  because  it  would  waste 
time,  the  instructor  in  the  rear  cockpit  would 
probably  coach  the  student  out  of  error  situations 
early,  and  it  would  affect  the  safety  of  flight. 
While  it  is  acknowledged  that  students  must  be 
allowed  to  see  the  results  of  their  mistakes  and 
that  a simulator  such  as  SAAC  is  probably  a good 
place  to  do  this,  there  are  times  when  multiple 
restarts  to  shape  initial  behaviors  also  may  be 
effective.  Similarly,  the  freeze  feature  was  rarelv 
used.  Rather  than  interrupting  an  engagement  to 
give  immediate  feedback,  instructors  preferred  to 
continue  and  run  another  trial.  It  would  seem  that 
segmenting  a maneuver  into  parts  and  discussing 
each  briefly  may  sometimes  be  more  effective.  The 
reluctance  to  use  other  instructional  features  on 
SAAC  was  felt  to  be  justified.  For  instant  e,  the 
use  of  record/ playback  would  not  be  expected  to 
aid  students  in  initial  experiences  with  BFM  until 
the  feature,  including  a visual  slaving  capability,  is 
reliable  and  easy  to  use. 

Other  issues  related  to  simulation  training  pro- 
cedures and  policy  are  discussed  in  two  recent 
HumRRO  reports  (Caro  1977b,  1977c).  Included 
are  issues  related  to  simulator  design,  training 
program  design,  instructor  training,  user  attitudes, 
and  administrative  policies. 

Platform  Motion  for  Air  Combat  S initiation 

The  results  of  the  current  experiment  leave  the 
question  of  the  need  for  platform  motion  for 
training  A/A  tasks  unanswered.  Recommendations 
from  other  reports  do  not  support  the  procure- 
ment of  platform  motion  for  fighter  training 
devices  (TFWC  TAC  ACES  11;  Rivers  & Van 
Arsdall,  1977).  It  is  recommended,  however,  that 
research  and  development  to  optimize  platform 
motion  drive  algorithms  continue  on  devices  cur- 
rently in  the  inventory.  More  information  is  also 
needed  on  how  motion  cues  are  perceived  and 
used.  With  regard  to  this,  the  recently  articulated 
distinction  between  “maneuver”  motion  cues  and 
“disturbance"  motion  cues  (Caro,  1977a)  may  be 
important.  Maneuver  motion  results  from  pilot 
initiated  (control  input)  changes  in  the  aircraft 
position.  Disturbance  motion  arises  outside  this 
control  loop  from  turbulence  or  from  airframe 
equipment  or  engine  component  failure. 

For  the  continuation  training  of  experienced 
pilots,  the  platform  motion  question  is  even  more 
difficult.  It  is  anticipated  that  it  would  be  veiy 
difficult  to  achieve  measureabie  changes  in  per- 
formance. The  subjective  opinions  of  operational 
pilots  may  be  the  only  data  currently  available  on 
which  to  base  these  decisions. 
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APPENDIX  A:  TRAINING  SYLLABUS 


TRAINING  SCHEDULE 


Training  Day 

21  to  24 
23  to  43 
44  to  49 


Initial  ACM  Academics 
SAAC  Training  Sorties  1 to  7 
BFM  Sorties  4 to  6b 


SAAC  1 Orientation 


Crew 
IP,  AC 


SAAC  Training 
Time 
.5 


Configuration 

D-33 


Sound -on-slide  briefing  (emphasize  safety),  cockpit  checkout  (emphasize  egress),  transition  maneuvers  (low 
AOA  roll,  lazy  eight,  loop),  advanced  handling  (acceleration  maneuver,  break  turn,  pitchback,  sliceback), 
out-of-control  recovery,  spin  recovery,  egress. 


SAAC-2  Orientation 


IP.  AC 


.5 


D-33 


Transition  maneuvers  (Immelmann,  cloverleaf),  advanced  handling  (sliceback,  pitchback),  tracking  exercise, 
fighting  wing,  free  engagement. 


SAAC-3  BFM  Training 


IP,  AC 


.7 


D-S5 


Tactical  formation  (AC  flies  #3  position),  set  up  on  perch,  estimate  range,  select  and  arm  AIM-9, 
acceleration  maneuver,  high  yo-yo,  launch  AIM-9,  quarter  plane  maneuver,  lag  roll,  separation  maneuver, 
fighting  wing,  free  engagement. 


SAAC-4  BFM  Training 


IP,  AC 


.7 


D-55 


Set  up  on  perch,  auto-acquisition  lock  on,  acceleration  maneuver,  high  yo-yo,  quarter  plane  maneuver, 
barrel  roll  maneuver,  Immelmann  turn,  counter  low  yo-yo,  counter  high  yo-yo  (executed  late),  fighting 
wing,  free  engagement. 

SAAC  5 BFM  Training  IP,  AC  1 .0  D 33 

Set  up  on  perch,  acceleration  maneuver,  high  yo-yo,  select,  arm  and  fire  gun.  quarter  plane  maneuver,  barrel 
roll  maneuver,  Immelmann  turn,  counter  high  yo-yo  (executed  early,  executed  properly),  fighting  wing, 
free  engagement. 

SAAC-6  BFM  Training  IP,  AC  1.0  D-66 

Tactical  formation,  tactical  intercept,  tune  and  arm  AIM-7,  launch  AIM-7,  stern  conversion,  launch  AIM-9, 
high  yo-yo,  barrel  roll,  Immelmann  turn,  counter  barrel  roll  attack,  break  turn,  free  engagement. 

SAAC-7  BFM  Evaluation  iP,  AC  1.0  D-66 

Tactical  formation,  set  up  on  perch,  acceleration  maneuver,  high  yo-yo,  barrel  roll  maneuver,  Immelmann 

turn,  defensive  counters,  launch  AIM-9,  launch  AIM-7,  fire  gun,  free  engagement. 


Aircraft  Sorties 


BFM  4 


Crew 

AC/IP,  AC/ IP 


Time 

1.3 


Configuration 

D-33 


Formation  takeoff,  tactical  formation,  auto-acquisition  lock-on,  high  yo-yo,  acceleration  maneuver,  quarter 
plane  maneuver,  separation  maneuver,  fighting  wing. 


BFM  6 


AC/IP,  AC/IP  1.3 


D-33 
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Formation  takeoff)  tactical  formation,  tracking  exercise.  High  yo*yo,  barrel  roll  maneuver,  Immelmann 
turn,  quarter  plane  maneuver,  separation  maneuver,  fighting  wing. 

BFM6A  AC/IP,  AC/PI  1.3  D 33 

Formation  takeoff,  tac.ical  formation,  barrel  roll  maneuver,  Immelmann  turn,  quarter  plane  maneuver, 
defensive  counters,  separation  maneuver,  fighting  wing. 

BFM6B  AC/IP,  AC/lP  13  D-33 

Formation  takeoif,  tactical  formation,  high  yo-yo,  quarter  plane  maneuver,  barrel  roll  maneuver, 
Immelmann  turn,  defensive  counters,  separation  maneuver. 

Configuration  Code 


Fuel  Configuration: 

A - Full  internal 

D — Full  internal/2  x 370  gallon  tanks  full 
Munitions  Configuration: 

33  — Clean,  inboard  pylons,  internal  gun. 

55  — AIM-9,  4 ea.,  internal  gun. 

66  — AIM-7,  4 ea.,  AIM-9,  4 ea.,  internal  gun. 


APPENDIX  B:  GRADING  CRITERIA 


Grade 

Unknown 

0 

1- 

1 

1+ 

2- 

2 

2+ 

3- 

3 

3+ 

4- 

4 


Definition 

Performance  not  observed  or  the  element  was  not  performed. 
Performance  indicates  a lack  of  ability  or  knowledge. 

Performance  is  safe  but  indicates  limited  proficiency.  Slakes  errors  of 
commission  or  omission. 


Performance  is  essentially  correct. 
Recognizes  and  corrects  errors. 


Performance  is  correct,  efficient,  skillful,  and  without  hesitation. 


Performance  reflects  an  unusually  high  degree  of  ability. 


APPENDIX  C QUESTIONNAIRES 


Instructions:  (Used  for  Both  IPs  and  Students) 

In  order  to  amplify  as  much  data  as  possible  concerning  the  use  of  platform  motion,  please  answer  the 
following  in  as  much  detail  as  you  feel  is  warranted  and  comment  on  other  areas  as  appropriate.  Thank  you. 

IP  Questionnaire: 

1.  How  did  the  SAAC  training  affect  student  performance  in  the  aircraft  on  BFM? 

2.  In  what  manner  and  to  what  extent  did  the  platform  motion  contribute  to  student  proficiency  in  the 
aircraft  on  BFM? 

Student  Questionnaire: 

1.  How  did  the  SAAC  training  (or  lack  thereof,  if  you  were  a non-SAAC  student)  affect  your  performance 
in  the  aircraft  on  BFM? 

2.  Please  give  spccifu  comments  on  the  use  of  the  motion  platform. 


> i 
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APPENDIX  D:  DATA  TABLES 
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Tabic  DI.  Mean  Initial  Rating  in  Simulator  and 

Significance  Probability 

Maneuver/Skill 

Qroup 

Probability 

No-Motlon 

Motion 

Acceleration  Maneuver 

4.50 

4.63 

.84 

High  Yo-Yo 

3.88 

3.38 

.45 

Quarter  Plane 

3.00 

3.50 

.53 

Barrel  Roll  Attack 

3.88 

3.00 

.23 

lmmelmann  Attack 

3.86 

3.38 

.56 

Lag  Roll 

3.75 

2.86 

.36 

Separation 

4.25 

5.25 

.28 

Tactical  Formation 

4.38 

4.75 

.59 

Set  up  on  Perch 

3.63 

4 13 

.55 

Defensive  Maneuvers 

4.25 

4.25 

1.00 

Descriptive  Commentary 

4.00 

3.75 

.77 

Range  Estimation 

4.38 

3.38 

.27 

Target  Acquisition 

6.14 

5 29 

.28 

Kept  Bogey  in  Sight 

5.50 

4.67 

.40 

Weapons  Parameters  Recognition 

5.00 

4.83 

.79 

Switchology 

5.40 

4.50 

.24 

Preparation 

6.29 

614 

.86 

Attitude 

7.71 

7.29 

.45 

Judgment 

5.86 

4.86 

.27 

Set  1 Maneuvers 

3.94 

3.92 

.96 

Set  2 Skills 

5.65 

5 02 

.05* 

•Significant  at  p < .10. 

Table  D2.  Mean  Final  Ratings  in  Simidator  and 

Significance  Probability 

Group 

Significance 

No-Motlon 

Motion 

Probability 

Acceleration  Maneuver 

5.13 

6.00 

.09’ 

High  Yo-Yo 

4.00 

5.38 

.21 

Quarter  Plane 

4.50 

6.13 

04* 

Barrel  Roll  Attack 

4.75 

5.75 

.21 

lmmelmann  Attack 

5.00 

5.38 

.62 

Lag  Roll 

+ 

+ 

+ 

Separation 

5.25 

5.88 

■51 

Tactical  Formation 

6.00 

5.88 

.87 

Set  up  on  Perch 

3.63 

3.38 

.81 

Defensive  Maneuvers 

5.38 

5.50 

.82 

Descriptive  Commentary 

6,25 

7.38 

.16 

Range  Estimation 

5.20 

5.50 

.62 

Target  Acquisition 

5.00 

6.33 

.53 

Kept  Bogey  in  Sight 

+ 

+ 

Weapons  Paremeters  Recognition 

3.75 

6.00 

.15 

Switchology 

4.25 

5.00 

.44 

Preparation 

6.17 

6.57 

.62 

Attitude 

7.50 

7.57 

.92 

Judgment 

5.00 

5.50 

.34 

Set  1 Maneuvers 

4.85 

5.45 

.03* 

Set  2 Skills 

5.58 

6.49 

.01* 

•Significant  at  p < .10. 
♦Insufficient  data. 
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,4f*. 


Table  OS.  Mean  Initial  Ratings  in  Aircraft  and  Significance  Probability 


Group 


Manauvar  /Skill 

N04AAC 

No  ^Motion 

Motion 

AoMMIty 

Acceleration  Maneuver 

4.67 

4.00 

4.13 

.51 

High  Yo-Yo 

4.33 

3.13 

3.88 

.26 

Quarter  Plane 

3.33 

3.13 

3.00 

.92 

Barrel  Roll  Attack 

3.67 

3.75 

3.00 

.57 

Immclmann  Attack 

3.83 

2.88 

2.75 

.38 

Lag  Roll 

4.17 

3.86 

3.88 

.93 

Separation 

3.33 

4.00 

3.13 

.72 

Tactical  Formation 

4.17 

3.75 

3.88 

.80 

Set  up  on  Perch 

4.33 

4.63 

4.00 

.77 

Defensive  Maneuvers 

4.50 

4.25 

4.75 

.72 

Descriptive  Commentary 

4.67 

5.13 

3.75 

.10* 

Range  Estimation 

4.33 

3.75 

4.75 

.21 

Target  Acquisition 

5.17 

5.50 

5.38 

.93 

Kept  Bogey  in  Sight 

4.67 

4.25 

5.13 

.69 

Weapons  Parameters  Recognition 

4.50 

3.25 

4.38 

.09* 

Switchology 

3.50 

2.88 

4.13 

.29 

Preparation 

6.17 

6.00 

5.00 

.29 

Attitude 

7.00 

7.25 

7.00 

.93 

Judgment 

6.50 

5.13 

5.13 

.20 

Set  1 Maneuvers 

4.03 

3.73 

3.64 

.29 

Set  2 Skills 

5.17 

4.79 

4.96 

.50 

‘Significant  at  p < .1 0. 


Table  D4.  Mean  Final  Ratings  in  Aircraft  and  Significance  Probability 

Ctfoop 

Slf  ntfloanca 
Probability 

NO'SAAC 

No-Motlon 

Motion 

Acceleration  Maneuver 

4.83 

4.50 

4.38 

.58 

High  Yo-Yo 

5.17 

4.75 

4.63 

.45 

Quarter  Plane 

5.50 

5.00 

4.00 

.04* 

Barrel  Roll  Attack 

4.67 

3.88 

4.25 

.47 

Immelmann  Attack 

4.83 

4.13 

4.63 

.54 

Lag  Roll 

- 

— 

- 

— 

Separation 

4.67 

4.86 

4.13 

.58 

Tactical  Formation 

4.67 

4.75 

4.75 

.97 

Set  up  on  Perch 

5.00 

5.38 

5.43 

.45 

Defensive  Maneuvers 

4.67 

4.25 

4.29 

.78 

Descriptive  Commentary 

5.00 

4.50 

4.83 

.90 

Range  Estimation 

4.50 

4.63 

4.75 

.87 

Target  Acquisition 

4.83 

5.43 

5.00 

.66 

Kept  Bogey  in  Sight 

4.33 

4.88 

5.13 

.61 

Weapons  Parameters  Recognition 

4.33 

4.43 

5.00 

.71 

Switchology 

4.33 

4.88 

4.88 

30 

Preparation 

7.02 

6 63 

5.88 

.35 

Attitude 

6.17 

6.63 

6.13 

.88 

Judgment 

6.50 

4.75 

• 00 

.15 

Set  1 Maneuvers 

4.89 

4.56 

',.47 

.06* 

Set  2 Skills 

5.22 

5.24 

5.19 

.98 

•Significant  at  p < .10. 
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Table  D5.  Mean  Overall  Rating  in  Aircraft  and  Significance  Probability 


Qrous 

No>SAAC 

No-Mollon 

Motion  Probability 

Acceleration  Maneuver 

5.01 

4.27 

4.11 

.01* 

High  Yo-Yo 

4.59 

4.18 

4.46 

.46 

Quarter  Plane 

4.27 

4.24 

3.63 

.50 

Barrel  Roll  Attack 

4.14 

3.97 

3.66 

.57 

lmmelmann  Attack 

4.21 

3.97 

3.73 

.66 

Lag  Roll 

4.29 

3.77 

3.91 

.78 

Separation 

4.64 

4.34 

3.73 

.33 

Tactical  Formation 

4.31 

4.14 

4.79 

.11 

Set  up  on  Perch 

4.56 

4.77 

4.65 

.89 

Defensive  Maneuvers 

4.45 

4.27 

4.67 

.55 

Descriptive  Commentary 

4.76 

4.76 

4.11 

.26 

Range  Estimation 

4.33 

4.54 

4.63 

.74 

Target  Acquisition 

4.60 

5.33 

5.33 

.21 

Kept  Bogey  in  Sight 

4.24 

4.99 

5.03 

.37 

Weapons  Parameters  Recognition 

4.38 

3.91 

4.71 

.19 

Switchology 

4.13 

4.22 

4.30 

.93 

Preparation 

6,26 

6.12 

5.56 

.45 

Attitude 

6.71 

6.91 

7.00 

.90 

Judgment 

5.44 

5.35 

5.38 

.99 

Set  1 Maneuvers 

4.44 

4.20 

4.13 

.12 

Set  2 Skills 

4.98 

5.12 

5.12 

.79 

•Significant  at  p < .10. 

Table  D6.  Significance  Probabilities  of  Initial 

vs  Final  Data 

Simulator 

Aircraft 

NO-Motlnn 

Mellon  No-SAAC  Nu-Motlon  Motion 

Acceleration  Maneuver 

.18 

.05* 

.72 

.42  .54 

High  Yo-Yo 

.86 

.07* 

.18 

.01*  .26 

Quarter  Plane 

.06* 

.01* 

.03* 

.01*  .15 

Barrel  Roll  Attack 

.28 

.01* 

.21 

.88  .03* 

lmmelmann  Attack 

.10* 

.03* 

.14 

.12  .02* 

Lag  Roll 

+ 

+ 

+ 

+ + 

Separation 

.37 

.39 

.20 

.44  .27 

Tactical  Formation 

.04* 

.12 

.41 

.06*  .06* 

Set  up  on  Perch 

1.00 

.45 

.26 

.35  .05* 

Defensive  Maneuvers 

.13 

.10* 

.77 

1.00  .38 

Descriptive  Commentary 

.03* 

.01* 

.67 

.59  .07* 

Range  Estimation 

.13 

.06* 

.79 

.12  1.00 

Target  Acquisition 

.46 

.34 

.69 

.93  .56 

Kept  Bogey  in  Sight 

f 

+ 

.74 

.56  1.00 

Weapons  Parameters  Recognition 

.14 

.28 

.77 

.20  .39 

Switchology 

.29 

.35 

.34 

.02*  .41 

Preparation 

.90 

.57 

.36 

.39  .27 

Attitude 

.71 

.66 

.38 

.40  .26 

Judgment 

.18 

.47 

1.00 

.64  .88 

Set  1 Maneuvers 

.01* 

.01* 

.01* 

.01*  .01* 

Set  2 Skills 

.83 

.01* 

.48 

.16  .37 

•Significant  at  p < .10. 
-•-Insufficient  data. 
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